CMU/IBM – Network Security and Anomaly Visualization

VAST 2011 Challenge
Mini-Challenge 2 – Computer Networking Operations at All Freight Corporation

Authors and Affiliations:

Qi Liao, Central Michigan University, qi.liao@cmich.edu  [PRIMARY contact]

Lei Shi, IBM Research - China, shllsh@cn.ibm.com

Weihong Qian, IBM Research - China, qianwh@cn.ibm.com

Su Zhong, IBM Research - China, suzhong@cn.ibm.com

Tool(s):

We mainly use a security visualization tool: Network Security and Anomaly Visualization (NSAV) to solve this challenge. NSAV was derived from ENAVis (Enterprise Network Activities Visualization) which was developed during Qi Liao’s Ph.D. study at the University of Notre Dame. In the graph visualization part, NSAV is based on the Prefuse toolkit developed by Jeff Heer. Prefuse was selected based its relatively easy customization and extendibility for interactive visual analysis.

Video:

Video link (Flash format, drag to the browser to display)

ANSWERS:

---

MC 2.1 Events of Interest: Using the new situation awareness display(s), what noteworthy events took place for the time period covered in the firewall, IDS and syslog logs? Which events are of concern from a security standpoint? Limit your answer to no more than five noteworthy events. For each event, at least one of the submitted screen shots must be relevant in your explanation of the event.

Noteworthy events discovered:

l The workstations 192.168.1.171-175 have severe security holes indicated by the Nessus network vulnerability scan report. These workstations also initiate various types of portscans, TCP communications anomalies and http attacks to the servers (192.168.1.*) and workstations (192.168.2.*).

In more details, 192.168.1.171-173 triggers a great amount of “snort_decoder” IDS records on the morning of 4/13/2011 to 192.168.2.11-138. The method is through the TCP packets with illegal window scale option length. This may result in an internal failure in the receiving machine and then lead to the attack. 192.168.1.171-173 also conduct port scans to several critical server machines including 192.168.1.2, 192.168.1.14, 192.168.1.6, during the three days.

On the other hand, 192.168.1.174-175 initiate a great amount of “snort_decoder” attacks on the morning 4/14/2011 to 192.168.l.* and 192.168.2.*. 192.168.1.174-175 conduct port scans to several critical server machines including 192.168.1.2, 192.168.1.14, 192.168.1.10, 192.168.1.11, 192.168.1.12, 192.168.1.13.

192.168.1.174-175 also trigger a few “http_inspect” type IDS records to 192.168.1.10, 192.168.1.11, 192.168.1.12, which are potential attacks to the web server.

Figure 1. Situations with workstation 192.168.1.171-175. 171-173 have similar anomaly patterns, including the bursty “snort_decoder” at the start and the continuous portscans. 174-175 have another type of anomaly pattern, with bursty “snort_decoder” at the second morning. All 5 workstations have severe system security hole during the time.

l The workstations 192.168.2.11-138 continue to portscan server machine 192.168.1.2, 192.168.1.14, 192.168.1.6. Some of the workstations, including 192.168.2.68, 2.154, 2.33, 2.55, 2.147, 2.92, 2.28, 2.143, 2.17, 2.56, 2.51 initiate fragmentation overlap attack to 192.168.1.2, 192.168.1.14, which could be a kind of teardrop attack.

Through analysis, these malicious behaviors could link to the attack from 192.168.1.171-173 on the morning of 4/13/2011 and the attack from 192.168.1.174-175 on the morning of 4/14/2011.

Figure 2. Situations with 192.168.2.11-138. These workstations are attacked on the morning of the first two days by “snort_decoder”. They continuously portscan 1.2/1.6/1.14 server machines.

l The DC/DNS/DHCP server 192.168.1.2 (primary), 192.168.1.14 (secondary) and mail server 192.168.1.6 encounter continuous portscans by both 192.168.2.11-138 (bots) and 192.168.2.171-175 (initiators).

192.168.1.2 / 192.168.1.14 get potential teardrop attacks by 192.168.2.68, 2.154, 2.33, 2.55, 2.147, 2.92, 2.28, 2.143, 2.17, 2.56, 2.51.

192.168.1.2 gets source IP address spoofing attack from the fake IP 192.168.7.151.

192.168.1.2 gets the portscan from external host 10.200.150.207.

There are several high-risk OS security events on 192.168.1.2, the primary data server, including security log cleared event, more than 15 times logon attempts using explicit credential, more than 50 times operations on account domain objects (user, group, group policy container, etc.), more than 20 times validation of the credentials and two logon fails and one computer account change.

Figure 3. Situations with 192.168.1.2 / 192.168.1.14 / 192.168.1.6. 1.2 and 1.14 are troubled with all-time portscans and some teardrop attacks. DC 1.2 also has various system account high-risk operations identified by os security logs.

l The workstations 192.168.1.10-13 get continuous snort_decoder/http_inspect attacks and portscan by 192.168.2.174-175.

192.168.2.174-175 launch attacks on the 135/445/3389 ports of 192.168.1.10-13.

There are also logon attempts with explicit credential on 192.168.1.13.

Figure 4. Situations with 192.168.1.10-13. 1.13 has logon attempts with explicit credential after attacked by 192.168.2.174-175 on 135/445/3389 ports.

l The external web server 172.20.1.5 is intruded by 10.200.150.201 at port 3389. Then 172.20.1.5 attacks the internal web server 192.168.1.5 at port 445 (well-known Windows system flows to execute arbitrary code). 192.168.1.5 later gets many OS security events on logon fails.

Figure 5. Situations with 172.20.1.5 and 192.168.1.5. 172.20.1.5 is the external web server connected with several external hosts (10.200.150.*). The web server is attacked from outside and then operated to attack the internal web server 192.168.1.5, leading to the logon fail security event on 192.168.1.5.

---

MC 2.2 Timeliness: For each event submitted in MC 2.1, how early in the course of the event would your display(s) enable a CNO team member to recognize that the event was noteworthy? For each event, specify the earliest moment of recognition as a timestamp and provide a screen shot at the earliest moment of recognition. Explain how the CNO team member had enough information to determine that the event warranted attention.

l The workstations 192.168.1.171-175 attack.

The detection could be as early as 2011-4-13 12:00:00, when bursty “snort_decoder” attacks in the morning just finished and the portscans issued by the workstations are spotted. CNO member could recognize the event from both the traffic graph that 192.168.2.171 (e.g.) has a star connection graph, and the bursty transmission of “snort_decoder” and portscans indicated in the right anomaly temporal panel.

Figure 6. Early detection of 192.168.1.171 event.

l The workstations 192.168.2.11-138 hijacked and portscan the servers (DDOS).

The detection could be as early as 2011-4-13 22:00:00, when the portscan to the servers have continued for a day.

Figure 7. Early detection of 192.168. 2.11-138 event.

l The DC/DNS/DHCP server 192.168.1.2/192.168.1.14/192.168.1.6 attacked.

The detection could be as early as 2011-4-13 23:00:00, when the first logon attempt with explicit credential has been issued at 192.168.1.2. Synthesizing the previous “fragment overlap” attack, the all-time portscans, and the changes to the account domain object, the CNO member could deduce the account security events have been happening on the server machines.

Figure 8. Early detection of 192.168. 1.2/1.14/1.6 event.

l The workstations 192.168.1.10-13 attacked.

The significance of the event could only be detected by the night of the last day, 2011-4-15 20:00:00, when two logon attempts with explicit credentials happened. Before that, there are several attacks to these workstations, but no further suspicious event is tracked.

Figure 9. Early detection of 192.168. 2.10-13 event.

l The external/internal web server 172.20.1.5/192.168.1.5 chain attacked and 192.168.1.5 got account security events.

The detection could happen by 2011-4-15 3:00:00, when the first logon fails are logged on 192.168.1.5. The CNO member could then trace back to the attack at port 445 by 172.20.1.5, and then relates the event to the intrusion by 10.200.150.201 at port 3389.

Figure 10. Early detection of 172.20.1.5 event.

We arrived at the results of MC2.1/2.1 by using the NSAV tool to diagnose the anomalies. The process takes three steps:

l Anomaly parsing

We extract the potential anomalies from the firewall, IDS, Nessus and OS security logs. A common format is defined for all the four types of anomalies:

<Timestamp>, <Host Machine IP (:port)>, <Anomaly Type>, <Detailed Description>

To parse firewall logs, we take a white list approach. We manually write a good rule set as below according to the manual interpretation of the AFC network operation policies and statuses. This takes us approximately 1 hour in total, including the initial setting and the iterative changes to the rule set.

Good firewall rule set:

10.200.150.0-255, *, 172.20.1.5, 80

10.200.150.0-255, *, 192.168.1.6, 25

192.168.2.10-250, *, 192.168.1.6, *

10.200.150.0-255, 25, 192.168.1.6, *

192.168.2.10-250, *, 10.200.150.0-255, 80

192.168.2.10-250, *, 192.168.1.2, *

192.168.2.10-250, *, 192.168.1.14, *

172.20.1.5, *, 192.168.1.2, *

172.20.1.5, *, 192.168.1.14, *

172.20.1.5, *, 192.168.1.4, *

*, *, 192.168.1.2, 53

*, 53, 192.168.1.2, *

192.168.1.0-255, *, 192.168.1.0-255, *

192.168.2.10-250, *, 192.168.2.10-250, *

192.168.1.2, 67, *, 67

The good rules are derived from the Acceptable Use Policy established for the All Freight network. The resulting firewall anomalies are the traffic not matched by all the rules. A sample of the firewall anomalies are below.

Firewall anomalies:

1302714937,192.168.7.151,firewall_s,192.168.7.151:123 --> 192.168.1.2:123

1302714937,192.168.1.2,firewall_d,192.168.7.151:123 --> 192.168.1.2:123

1302715962,192.168.7.151,firewall_s,192.168.7.151:123 --> 192.168.1.2:123

1302715962,192.168.1.2,firewall_d,192.168.7.151:123 --> 192.168.1.2:123

1302717498,192.168.7.151,firewall_s,192.168.7.151:123 --> 192.168.1.2:123

1302717498,192.168.1.2,firewall_d,192.168.7.151:123 --> 192.168.1.2:123

1302717754,192.168.7.151,firewall_s,192.168.7.151:123 --> 192.168.1.2:123

1302717754,192.168.1.2,firewall_d,192.168.7.151:123 --> 192.168.1.2:123

For the IDS, all the records are kept as anomalies as the IDS already did the filtering process. For each record, still the timestamp, host IP, anomaly type and description are extracted. A sample can be found below.

IDS log anomalies:

1302869301,192.168.2.153,portscan_s,TCP Portscan to 192.168.1.2

1302869301,192.168.1.2,portscan_d,TCP Portscan from 192.168.2.153

1302869317,192.168.2.73,portscan_s,TCP Decoy Portscan to 192.168.1.6

1302869317,192.168.1.6,portscan_d,TCP Decoy Portscan from 192.168.2.73

1302869336,192.168.2.105,portscan_s,TCP Portsweep to 192.168.1.14

1302869336,192.168.1.14,portscan_d,TCP Portsweep from 192.168.2.105

1302869336,192.168.2.105,portscan_s,TCP Portscan to 192.168.1.14

1302869336,192.168.1.14,portscan_d,TCP Portscan from 192.168.2.105

1302869349,192.168.2.153,portscan_s,TCP Portsweep to 192.168.1.2

1302869349,192.168.1.2,portscan_d,TCP Portsweep from 192.168.2.153

1302869366,192.168.2.95,portscan_s,TCP Decoy Portscan to 192.168.1.6

For the OS security logs, since all the security events are included in the file, we have conducted a statistical filtering to discover the abnormal events. To do that, we first count all the events in the three-day logs and find out the total number of events to be 12. Then we manually check the few events with less occurrences and finally list 6 of them to be suspicious OS events. Then once again, we parse the security logs and extract all the occurrence of these 6 event types. A sample result can be found below.

OS security log anomalies:

1302893699,192.168.1.2,Directory Service Access,An operation was performed on an object.

1302891600,192.168.1.5,Logon,An account failed to log on.

1302890700,192.168.1.5,Logon,An account failed to log on.

1302890099,192.168.1.2,Directory Service Access,An operation was performed on an object.

1302889800,192.168.1.5,Logon,An account failed to log on.

1302888900,192.168.1.5,Logon,An account failed to log on.

1302888000,192.168.1.5,Logon,An account failed to log on.

1302886499,192.168.1.2,Directory Service Access,An operation was performed on an object.

1302882899,192.168.1.2,Directory Service Access,An operation was performed on an object.

1302880799,192.168.1.5,Logon,An account failed to log on.

For Nessus reports, similarly we extract the records with security holes. The results are somewhat simple: 192.186.1.171-175 have severe security holes.

l Graph construction

The topology of network connectivity graphs are constructed directly from the firewall log data, which is similar to NetFlow data, where each source IP address and port number has established connection states with a destination IP and port. For concise purpose, only host level (IP) connections are used as edges. Timestamps are recorded as one of the filtering mechanisms so that when investigators adjust the time slider in the visualization tool, the graphs can be dynamically filtered and generated interactively for easier exploration.

The network graphs are then combined with other types of security log data such as intrusion detection system (IDS) – snort, OS security event log and Nessus vulnerability scan data via events chaining based on the logical order of system timestamps. As a subset of flow data, security events as anomalies are rendered and meshed up with the flow graphs. Different icons (orange representing source and grey representing destination of anomalies) are used for easy visualization. Generally speaking, the more anomaly icons a node gets, the more abnormal it is. The notion of anomaly cluster can be quickly overviewed in our tool as shown in the screenshots above. Finally, scatter views of anomaly events straightforwardly present the correlations among the security anomalies on different nodes for human analysis.

l Visualization and exploration

Figure 1-5 shows the NSAV visualization in our case. In the main view in the top-left, the tool shows the graph view of the host traffic connections extracted from firewall logs. Initially, an overview of all the connections are presented, the user could also click on one node to show the graph central to the node. In the right panel of the interface, the anomalies happened to the selected nodes are shown, with a textual list view in the top and a detailed temporal view in the bottom. In the bottom of the entire interface, there is a time slider. The user could manually select a time range and go deep to the graph and anomalies in this time.

---

MC 2.3 Recommendations: What are the implications of the events discovered in MC 2.1? What report should the CNO give to the CEO and/or what actions should the CNO take to improve security?

First, the events discovered in MC 2.1 implies that the nodes 192.168.1.171-175 are sources of security attacks due to the operating system security holes so arbitrary code can be executed on the remote host. One recommendation that CNO should give to the CEO is to patch and fix the security holes on these nodes (1.171-1.175). Policy should be made to ensure all hosts running on the All Freight network should be periodically updated and unpatched machines should not be connected to the cooperate network. For other packet-based attacks, routers can be configured to drop packets with invalid options.

While these suggestions may be helpful, there is no panacea for security problem. The most practical and important approach is being able to detect and find the sources/causes of security violations and anomalies if they happen. Visual analytic tools such as the one developed and presented in this article can be extremely helpful to network operators and administrators at enterprise networks like All Freight.